Neural Networks with À La Carte Selection of Activation Functions

نویسندگان

چکیده

Activation functions (AFs), which are pivotal to the success (or failure) of a neural network, have received increased attention in recent years, with researchers seeking design novel AFs that improve some aspect network performance. In this paper we take another direction, wherein combine slew known into successful architectures, proposing three methods do so beneficially: (1) generate AF architectures at random, (2) use Optuna, an automatic hyper-parameter optimization software framework, Tree-structured Parzen Estimator (TPE) sampler, and (3) Optuna Covariance Matrix Adaptation Evolution Strategy (CMA-ES) sampler. We show all often produce significantly better results for 25 classification problems when compared standard composed ReLU hidden units softmax output unit. TPE sampler emerged as best architecture-producing method.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Death à la carte

Delegates arrived at the hotel on a rainy Sunday evening where they were immediately made welcome and encouraged to relax over dinner while chatting to other EWCD (European Workshop on Cell Death) participants. The EWCD is a bi-annual conference that focuses on all aspects of cell death. Each conference is held in a different unique location, with this year’s being no exception. The six-day mee...

متن کامل

Concurrent Objects à la Carte

Services are autonomous, self-describing, technology-neutral software units that can be described, published, discovered, and composed into software applications at run-time. Designing software services and composing services in order to form applications or composite services requires abstractions beyond those found in typical object-oriented programming languages. In this paper, we explore a ...

متن کامل

Fishing history à la carte

The decline in many marine fisheries is well documented but there is less information about the scale and timing of their demise. Researchers are now beginning to search for such information in the hope that it may help reveal how different fisheries might best be conserved or restored. One novel source has been seafood prices collected from US restaurant menus dating to the 1850s, which are he...

متن کامل

Polarization response of nanowires à la carte

Thanks to their special interaction with light, semiconductor nanowires have opened new avenues in photonics, quantum optics and solar energy harvesting. One of the major challenges for their full technological deployment has been their strong polarization dependence in light absorption and emission. In the past, metal nanostructures have been shown to have the ability to modify and enhance the...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: SN computer science

سال: 2021

ISSN: ['2661-8907', '2662-995X']

DOI: https://doi.org/10.1007/s42979-021-00885-1